When are Graphical Causal Models not Good Models?

The principle of Kolmogorov Minimal Sufficient Statistic (KMSS) states that a model should capture all regularities of the data. The conditional independencies following from the causal structure of the system are the regularities incorporated in a graphical causal model. We prove that for joint probability distributions, the KMSS is described by the Directed Acyclic Graph (DAG) of the minimal Bayesian network if this results in an incompressible description. We prove that a Bayesian network that is the KMSS is faithful. In that case it can be learned from observations and modularity is the most plausible hypothesis. From modularity follows the ability to predict the effect of interventions. On the other hand, if the minimal Bayesian network is compressible, and thus not the KMSS, the above implications cannot be guaranteed. When the non-minimality of the description is due to the compressibility of an individual Conditional Probability Distribution (CPD), the true causal model is an element of the set of minimal Bayesian networks and modularity is still plausible. Faithfulness cannot be guaranteed though. When the concatenation of the descriptions of the CPDs is compressible, the true causal model is not necessarily an element of the set of minimal Bayesian networks. Also modularity may become implausible. This suggests that either there is a kind of meta-mechanism governing some of the mechanisms or either a single mechanism responsible for setting the state of multiple variables.